build(maru): wire :maru:* subprojects into the linea-monorepo build#3126
Conversation
Workflows will be re-added at the monorepo root as .github/workflows/maru-*.yml with paths filters in a later commit (mirrors the tracer import pattern from PR #1852). Maru's dependabot.yml, pull_request_template.md, and ISSUE_TEMPLATE/ are dropped — the monorepo has its own equivalents.
The maru subprojects will be wired directly into zkevm/settings.gradle as :maru:* includes in a later commit. There is only one Gradle settings.gradle per build, so maru's own settings.gradle must go.
A Gradle build supports only one buildSrc. Maru's five convention plugins
(maru.kotlin-{common-minimal,common,library,library-minimal,application}-conventions)
are functionally equivalent to zkevm's existing net.consensys.zkevm.kotlin-*
family — same five conventions, with zkevm's being slightly richer (catalog-driven
versions, --enable-native-access JVM arg, consistent dependency resolution,
annotation-default-target).
Maru subprojects' 'apply plugin: maru.kotlin-X-conventions' lines will be
switched to 'apply plugin: net.consensys.zkevm.kotlin-X-conventions' in a
later commit.
…entions Required for the imported maru subprojects which depend on: - io.libp2p:jvm-libp2p (cloudsmith) — used by maru's p2p, crypto, utils - com.github.multiformats:* (jitpack) — libp2p transitive - com.splunk.* — parity with maru's previous build setup Each repository is content-filtered to the specific group(s) it serves so resolution doesn't unnecessarily query these endpoints for unrelated artifacts.
Adds 22 includes mirroring maru's original settings.gradle layout, with paths re-rooted under :maru:* (e.g. ':app' → ':maru:app', 'jvm-libs:extensions' → 'maru:jvm-libs:extensions'). The 'include maru' line ties maru's existing build.gradle to the :maru subproject; the next commit gutts that file to drop dependencies on the removed maru/buildSrc + gradle/versions.gradle.
The io.spring.dependency-management plugin is added with 'apply false' on the
root project's plugin classpath so maru subprojects can apply it via
'alias(libs.plugins.dependencyManagement)' in their own plugins{} block.
Maru relies on dependencyManagement for BOM-driven version resolution from
maru/gradle/versions.gradle (unversioned 'implementation' declarations
resolved via the dependencyManagement{} DSL). A follow-up commit folds
versions.gradle into the catalog, after which most subprojects can drop the
plugin.
…ct paths
- Switch every maru subproject's apply-plugin line from 'maru.kotlin-X-conventions'
to 'net.consensys.zkevm.kotlin-X-conventions'. Maru's conventions were
removed in an earlier commit; zkevm's same-named conventions are functional
equivalents.
- Add 'alias(libs.plugins.dependencyManagement)' to each maru subproject's
plugins{} block. Maru relies on this for BOM-driven version resolution from
maru/gradle/versions.gradle (unversioned 'implementation' declarations
resolved via the dependencyManagement{} DSL).
- Apply maru/gradle/versions.gradle directly in each subproject (after the
plugins{} block). A parent-level subprojects{ apply from } closure doesn't
work because the parent's closure evaluates before the subproject's plugins
block has applied dependencyManagement.
- Rewrite all 'project(":<name>")' refs in maru's subprojects to
'project(":maru:<name>")'. Maru's internal cross-references like
'project(":api")' previously resolved from maru's own rootProject; in the
monorepo they need the full :maru:* path.
- Gut maru/build.gradle to a minimal parent that only layers the SPDX
license-header + Java importOrder Spotless rules on top of the monorepo's
base config. All other configuration (errorprone, jacoco, JaCoCo aggregation,
test args, all the allprojects/subprojects boilerplate) is now provided by
the monorepo's root build.gradle and zkevm/buildSrc conventions.
- Add Vert.x + micrometer + linea metrics/micrometer/vertx-helper deps to
maru/app/build.gradle. These were previously bundled by maru's
kotlin-application-conventions plugin (deleted with maru/buildSrc); zkevm's
lighter kotlin-application-conventions doesn't bundle them, so they go
explicit in app/build.gradle.
Build is not yet green: build.linea.internal:* deps still resolve as remote
artifacts via versions.gradle's lineaPackagesVersion default. The next commit
folds gradle/versions.gradle into the catalog and substitutes those refs to
local zkevm subprojects.
…oject refs
Maru's subprojects depended on 13 build.linea.internal:* artifacts via
versions.gradle's dependencyManagement BOM pins. Those artifacts are products
of the zkevm-monorepo build itself, so in the monorepo we wire them as project
references instead of remote dependencies. Mapping:
build.linea.internal:metrics → :jvm-libs:linea:core:metrics
build.linea.internal:micrometer → :jvm-libs:linea:metrics:micrometer
build.linea.internal:vertx-helper → :jvm-libs:generic:vertx-helper
build.linea.internal:json-rpc → :jvm-libs:generic:json-rpc
build.linea.internal:futures → :jvm-libs:generic:extensions:futures
build.linea.internal:kotlin → :jvm-libs:generic:extensions:kotlin
build.linea.internal:domain-models → :jvm-libs:linea:core:domain-models
build.linea.internal:interfaces → :jvm-libs:linea:clients:interfaces
build.linea.internal:linea-contract-clients → :jvm-libs:linea:clients:linea-contract-clients
build.linea.internal:web3j-extensions → :jvm-libs:linea:web3j-extensions
build.linea.internal:file-system → :jvm-libs:linea:testing:file-system
build.linea.internal:logging → :jvm-libs:generic:logging
build.linea.internal:long-running-service → :jvm-libs:linea:core:long-running-service
Two small source fixes were needed for compilation under the monorepo:
- QbftFinalStateAdapter.kt:58: '.equals(localAddress)' → '== localAddress'
(Kotlin 2.x flags the explicit .equals() call as deprecated; zkevm's
-Werror would otherwise fail the build).
- ConsensusMetrics.kt:70: drop 'percentileBuckets = [...]' parameter from
metricsFacade.createHistogram(). zkevm's :jvm-libs:linea:core:metrics
interface does not yet expose that parameter (newer maru-side API).
publishPercentileHistogram=true is still set, so histograms still emit
percentiles with default buckets.
Build is green: './gradlew :maru:app:compileKotlin -x spotlessCheck' succeeds.
gradle/versions.gradle is still applied per-subproject and still pins maru-only
external deps (Discovery, jvm-libp2p, Javalin, http4k, etc.); the next commit
folds those into gradle/libs.versions.toml and removes versions.gradle.
Each maru subproject previously applied the io.spring.dependency-management
plugin and `apply from: maru/gradle/versions.gradle`, which used Spring DM
BOM imports (vertx-stack-depchain, besu, slf4j-bom) to govern transitive
versions. This duplicated zkevm's own version source-of-truth and pulled
Vert.x 4 in via the Besu BOM, conflicting with the local Vert.x 5
:jvm-libs:generic:vertx-helper at runtime.
Move every maru subproject onto zkevm's gradle/libs.versions.toml:
- add maru-only entries (discovery, jvm-libp2p, javalin, http4k, okhttp,
pkts-core, rocksdbjni, docker-compose-rule, kotlinx-datetime)
- reuse existing aliases (jackson, guava, hoplite, picoli, tuweni, teku,
web3j, log4j, awaitility, etc.) via "${libs.versions.X.get()}"
interpolation, matching the coordinator/* pattern
- leave org.hyperledger.besu* and io.vertx:* versionless: aligned by
root build.gradle's eachDependency rule and kotlin-common-conventions'
vertx-stack-depchain platform respectively
- drop libs.plugins.dependencyManagement and `apply from versions.gradle`
from all 21 subprojects
- delete maru/gradle/versions.gradle and the stale
maru/gradle/libs.versions.toml left over from the import
Also: replace deprecated Hash.toString() (-Werror failure post-fold) with
.bytes.toHexString() in BesuTransactionsHelper.
Port maru's standalone-repo workflows to live at the monorepo root,
mirroring the established coordinator-*/tracer-*/linea-besu-* pattern:
- maru-testing.yml (unit + integration tests, codecov)
- maru-build-and-publish.yml (docker image build via shared composite)
- maru-chaos-testing.yml (workflow_dispatch only for now)
- maru-e2e-tests.yml (workflow_dispatch only for now)
- maru-smoke-tests.yml (workflow_dispatch only for now)
- maru-release.yml (triggered by maru-v* tags)
Wire maru into the orchestrator workflows:
- main.yml: add `maru` paths-filter and pipe `maru_changed`/
`maru_image_tagged` into check-and-tag-images, testing, and
build-and-publish
- testing.yml: add maru_changed input + maru job
- build-and-publish.yml: add maru_changed/maru_image_tagged inputs
and maru job
- reuse-check-images-tags-and-push.yml: add maru_changed input,
image_tagged_maru output, and the corresponding image-tag check
and push steps
Transformations applied per workflow:
- Gradle invocations rewritten to :maru:* task paths
- setup-java + setup-gradle replaced by ./.github/actions/setup-
java-and-gradle composite
- DOCKER_ORG_NAME / DOCKER_REPO_TOKEN renamed to DOCKERHUB_USERNAME /
DOCKERHUB_TOKEN to share zkevm's existing Docker secrets
- Release tag pattern changed from `v*` to `maru-v*` to avoid
colliding with other components in the monorepo
- CHANGELOG and tar paths rebased to maru/
Deferred:
- security-code-scanner.yml (needs SECURITY_SCAN_METRICS_TOKEN and
APPSEC_BOT_SLACK_WEBHOOK secrets provisioned first)
- PR-time wiring for chaos / smoke / e2e (workflows exist but are
only reachable via workflow_dispatch in this PR)
- Aggregated jacoco upload (per-subproject .exec artifacts uploaded
as jacoco-maru-exec; monorepo's jacoco-report job not yet
consuming this artifact)
…mpat
Maru integration tests use Besu 26.5's acceptance-tests-dsl
(ThreadBesuNodeRunner, Cluster) to spin up real Besu nodes in-JVM. Besu
26.5's bytecode is bound to:
- Removed Vert.x 4 internal types: io.vertx.core.impl.ConcurrentHashSet,
used by PeerPermissionsDenylist / MaintainedPeers /
PrometheusMetricsSystem / PrometheusGuavaCache$Context /
AccessLocationTracker$AccountAccessList.
- Removed Vert.x 4 methods on still-present types:
Vertx.nettyEventLoopGroup() (Vert.x 5 removed it entirely).
- The pre-1.13 Micrometer Prometheus package
io.micrometer.prometheus.PrometheusMeterRegistry (renamed to
io.micrometer.prometheusmetrics.PrometheusMeterRegistry in 1.13+).
Class-level shims can cover removed types but not removed methods on
interfaces we don't own, so maru's test runtime must actually be on Vert.x 4
+ Micrometer 1.12.
Force in :maru's subprojects { ... } block:
- all io.vertx:* (except vertx-stack-depchain, to avoid colliding with
zkevm convention's strict 5.0.10 platform pin) -> 4.5.24
- all io.micrometer:* -> 1.12.13
And swap :maru:app's vertx-helper from the local Vert.x 5
:jvm-libs:generic:vertx-helper to the published Vert.x 4-era
build.linea.internal:vertx-helper:0.0.1-v20260327160734 so its bytecode
references resolve cleanly against the forced runtime.
Verified:
- :maru:jvm-libs:test-utils:test: 32 / 32 pass (was 23 / 32)
- :maru:app:integrationTest: 38 / 39 pass (was 0 / 39); 1 flake passes
on retry, unrelated to Vert.x
Drop the entire eachDependency block and switch back to
:jvm-libs:generic:vertx-helper once Besu moves to Vert.x 5.
Missed by the catalog-fold conversion; caught by ./gradlew spotlessCheck on the monorepo-wide pass.
linea-besu Changelog Preview (informational)[Unreleased] diff (commits touching
|
tx-exclusion-api Changelog Preview (informational)[Unreleased] diff (commits touching
|
coordinator Changelog Preview (informational)[Unreleased] diff (commits touching
|
prover Changelog Preview (informational)[Unreleased] diff (commits touching
|
postman Changelog Preview (informational)[Unreleased] diff (commits touching
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if-no-files-found: ignore | ||
| - name: Upload test results to Codecov | ||
| if: ${{ !cancelled() && env.CODECOV_TOKEN != '' }} | ||
| uses: codecov/test-results-action@v1 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3126 +/- ##
=========================================
Coverage 77.25% 77.25%
- Complexity 6991 7004 +13
=========================================
Files 1121 1118 -3
Lines 44418 44394 -24
Branches 5335 5343 +8
=========================================
- Hits 34314 34298 -16
+ Misses 8751 8730 -21
- Partials 1353 1366 +13
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
CI was failing with a circular dependency between buildNeeded tasks:
:maru:config:buildNeeded
\--- :maru:core:buildNeeded
\--- :maru:serialization:buildNeeded
+--- :maru:config:buildNeeded
+--- :maru:core:buildNeeded
\--- :maru:crypto:buildNeeded
\--- :maru:core:buildNeeded
The cycle exists because :maru:core's testFixtures depend on :maru:serialization
(+ observability, p2p, crypto), and :maru:serialization depends on :maru:core's
main jar via implementation. That's a clean linear graph under normal `build`,
but `buildNeeded` walks testFixtures too and trips on the cycle.
Standalone maru's CI used `./gradlew build` which doesn't recurse via
`buildNeeded`; our port copied coordinator-testing's pattern instead. Move maru
back to the build-aggregate approach by:
- Wiring :maru's existing root `build` task (auto-registered by zkevm's
`allprojects { apply plugin: 'java' }`) to depend on every :maru:* subproject's
`build`. Each subproject's `build` -> `check` -> (for :maru:app) integrationTest.
- Switching maru-testing.yml from `:maru:app:buildNeeded` to `:maru:build`.
Locally `./gradlew :maru:build` runs every :maru:*:test (passes) plus
:maru:app:integrationTest (passes on isolated retry; remaining flakes are
parallel-execution timing issues already known in standalone maru).
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 6fbf44e. Configure here.
| maru/**/build/reports/tests/ | ||
| - name: Upload Jacoco execution data | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 |
There was a problem hiding this comment.
Inconsistent upload-artifact version: @v7 vs @v4
Medium Severity
The Jacoco upload step uses actions/upload-artifact@v7 while the test reports step on line 55 uses actions/upload-artifact@v4. Every other workflow in this PR and the monorepo consistently uses @v4. Version 7 of this action does not exist as a released major version — the latest is v4. This will cause the "Upload Jacoco execution data" step to fail, preventing Jacoco coverage artifacts from being collected.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 6fbf44e. Configure here.
| - name: Build dist | ||
| run: | | ||
| ./gradlew :maru:app:installDist | ||
| - name: Upload distribution artifact |
There was a problem hiding this comment.
Why do we need to upload it?
There was a problem hiding this comment.
I just aligned it with what we have for coordinator. However this can be removed and integrate with the new release process. This can be done in a follow up PR.
| // block. Maru continues to use io.spring.dependency-management for its | ||
| // BOM-driven version resolution until maru/gradle/versions.gradle is | ||
| // folded into the catalog (follow-up commit). | ||
| alias(libs.plugins.dependencyManagement) apply false |
There was a problem hiding this comment.
This raises the question: Shall the monorepo use io.spring.dependency-management or maru build be aligned to monorepo?
@Filter94 what's your take on this?
Does io.spring.dependency-management pay off to the whole monorepo?
There was a problem hiding this comment.
what's your take on this?
I don't think it pays off. I think it was pulled together with Besu dependencies, but since then, I believe even in Besu they got rid of it. I'd treat io.spring.dependency-management as a tech debt
There was a problem hiding this comment.
Will followup in separate PR.
| content { includeGroup('com.github.multiformats') } | ||
| } | ||
| maven { | ||
| url 'https://splunk.jfrog.io/splunk/ext-releases-local' |
There was a problem hiding this comment.
what do we use this for?
There was a problem hiding this comment.
I didn't look into it, thses repositories were in maru so I moved them here while merging. After analyzing now, I see that It's introduced because maru's tests pull in besu-acceptance-tests-dsl, which transitively brings in besu-app, and Besu ships an optional Splunk HEC log appender. I can check if it can be removed in a followup PR
| // resolves without conflict. | ||
| // :maru already has a `build` task from zkevm root's `allprojects { apply plugin | ||
| // 'java' }`. Wire it up to also depend on every :maru:* subproject's build, so | ||
| // `./gradlew :maru:build` becomes the CI entry point. |
There was a problem hiding this comment.
Avoiding
buildNeededis deliberate: maru's :maru:core has testFixtures that
// depend on :maru:serialization, which in turn depends on :maru:core's main jar.
// That's a clean linear dep in normal builds butbuildNeededwalks testFixtures
// too and trips on the cycle (config -> core -> serialization -> config). Per-
// subprojectbuilddoesn't traverse cross-project test deps that way, so it
// resolves without conflict.
we must fix this circular dependency. We should create a maru:core-{interfaces} module that has the interfaces and modules to avoid circular dependencies.
There was a problem hiding this comment.
Yes, the goal for now is to have build and CI integrated. I will create a followup ticket so that this can be addressed in a separate PR
| name: maru-distribution | ||
| path: maru/app/build/install/app/ | ||
| retention-days: 1 | ||
| - name: Build and publish Docker image |
There was a problem hiding this comment.
Maybe we should instead integrate it with the new release process
There was a problem hiding this comment.
I just aligned it with what we have for coordinator. However this can be removed and integrate with the new release process. This can be done in a follow up PR.
| # because :maru:core's testFixtures depend on :maru:serialization | ||
| # which depends on :maru:core's main jar -- linear under `build` but a | ||
| # cycle under `buildNeeded` (which walks testFixtures). | ||
| ./gradlew -V :maru:build --rerun-tasks |
There was a problem hiding this comment.
I think we're better off with --rerun-tasks disabled in the Monorepo
There was a problem hiding this comment.
Yes, however this is only triggered on a maru change so "--rerun-tasks" does not result in high inefficiency. I will create a followup PR that address the cleanup.
| tags = listOf(Tag("role", role)), | ||
| publishPercentileHistogram = true, | ||
| percentileBuckets = listOf(.5, .8, .95, .99), | ||
| // NOTE: zkevm's createHistogram in :jvm-libs:linea:core:metrics does not yet expose |
There was a problem hiding this comment.
Ok, will remove this comment in a follow up
| if (subproject.plugins.hasPlugin('com.diffplug.spotless')) { | ||
| subproject.spotless { | ||
| kotlin { | ||
| licenseHeaderFile( |
There was a problem hiding this comment.
I don't think we need a separate license file for Maru
There was a problem hiding this comment.
ok, I don;t think we have a common license in monorepo. eg: coordinator files don't have the license header, tracer uses its own license check.
5 participants
#2540
Summary
Companion to #3100 (the history-only import). That PR landed
maru/under linea-monorepo with full git history but the subtree didn't yet build. This PR makes it build.12 logical commits, separable for review:
.github/,settings.gradle, andbuildSrc/.kotlin-common-minimal-conventions; register:maru:*subprojects in monoreposettings.gradle; rewriteapply plugin:lines tonet.consensys.zkevm.kotlin-*-conventions; rewriteproject(':<name>')refs toproject(':maru:<name>'); substitutebuild.linea.internal:*deps with local:jvm-libs:*subprojects; foldmaru/gradle/versions.gradleintogradle/libs.versions.toml..github/workflows/maru-*.yml(chaos, e2e, smoke, build-and-publish, release, testing) and wiremaru_changedintomain.yml,testing.yml,build-and-publish.yml,reuse-check-images-tags-and-push.yml.acceptance-tests-dslis bytecode-bound to removed Vert.x 4 types (io.vertx.core.impl.ConcurrentHashSet), removed methods (Vertx.nettyEventLoopGroup), and the pre-1.13io.micrometer.prometheus.*package, none of which can be supplied by class-level shims. Maru's:maru:appswaps:jvm-libs:generic:vertx-helperfor the published Vert.x-4-erabuild.linea.internal:vertx-helper:0.0.1-v20260327160734(resolves from Maven Central). Drop this once Besu moves to Vert.x 5.Verification (local)
:maru:*:compileKotlin(main, test, integrationTest, acceptanceTest sources): all green.:maru:jvm-libs:test-utils:test: 32/32 pass.:maru:app:integrationTest: 38/39 pass + 1 known QBFT round-change flake (passes on retry; unrelated to anything in this PR)../gradlew spotlessCheck(whole monorepo): green.git log --follow maru/<file>andgit blame maru/<file>: history attribution preserved post-import.Deferred to follow-up PRs
security-code-scanner.ymlport — needsSECURITY_SCAN_METRICS_TOKENandAPPSEC_BOT_SLACK_WEBHOOKGitHub secrets provisioned in linea-monorepo first.maru-chaos-testing.yml,maru-smoke-tests.yml,maru-e2e-tests.yml— workflow files exist but are only reachable viaworkflow_dispatch. Standalone maru'smain.ymlwired chaos (on path changes) and smoke (every PR); we can replicate once we're satisfied the base testing path is stable.**/build/jacoco/*.execis uploaded asjacoco-maru-exec(coordinator-style), but the monorepo'sjacoco-reportjob doesn't yet consume it.:jvm-libs:generic:vertx-helper(Vert.x 5) back — once Besu 26.6+ moves to Vert.x 5, drop the Vert.x 4 / Micrometer 1.12 force inmaru/build.gradleand switch maru'sappback to the local vertx-helper.Notes for review
maru-build-and-publish.yml:DOCKER_ORG_NAME→DOCKERHUB_USERNAME,DOCKER_REPO_TOKEN→DOCKERHUB_TOKEN(reuses the monorepo's existing secrets).v*→maru-v*to avoid colliding with other components in the monorepo."${libs.versions.X.get()}"interpolation, matching thecoordinator/*pattern. Maru-only catalog entries added:discovery,dockerComposeRule,http4k,javalin,jvmLibp2p,kotlinxDatetime,lineaPackages,okhttp,pktsCore,rocksdbjni.Test plan
maru-testingjob.coordinator-*,linea-besu-*,tracer-*, etc. — should be unaffected since maru changes are subproject-scoped).git log --followon a maru file from the merged PR view.Note
Medium Risk
Moderate risk because it changes core Gradle wiring and CI pipelines, including new dependency/version pinning (Vert.x/Micrometer) that could affect build/test resolution or image publishing behavior.
Overview
Enables the imported
maru/subtree to build and test inside the monorepo by registering all:maru:*projects in rootsettings.gradle, switching maru modules to the monoreponet.consensys.zkevmGradle convention plugins, and replacing maru’s prior dependency-management setup with versions fromgradle/libs.versions.tomlplus new maru-specific entries.Updates maru’s build to work with monorepo tooling: introduces an aggregate
:maru:buildentry point, adds required artifact repositories for maru dependencies, and adds maru-scoped dependency pinning (force Vert.x 4 and Micrometer 1.12) to maintain Besu 26.5 runtime compatibility.Integrates maru into CI/image flows: adds new reusable workflows (
maru-testing,maru-build-and-publish, plus maru e2e/smoke/chaos/release updates), wiresmaru_changedthroughmain.yml,testing.yml,build-and-publish.yml, andreuse-check-images-tags-and-push.yml, and removes maru’s standalone.github/workflows/templates andbuildSrcin favor of monorepo equivalents.Reviewed by Cursor Bugbot for commit 6fbf44e. Bugbot is set up for automated code reviews on this repo. Configure here.